<html>
<head><meta charset="utf-8"><title>compiling the compiler statically · t-compiler · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/index.html">t-compiler</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/compiling.20the.20compiler.20statically.html">compiling the compiler statically</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="187611470"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/compiling%20the%20compiler%20statically/near/187611470" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/compiling.20the.20compiler.20statically.html#187611470">(Feb 07 2020 at 03:56)</a>:</h4>
<p>I'd like to consider shipping <code>rustc</code> as a single statically linked executable. This would mean that compiler plugins and compiler backends (as they exist today) would no longer be supported without a custom built compiler. I'm wondering who the users of these features are.</p>



<a name="187614086"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/compiling%20the%20compiler%20statically/near/187614086" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/compiling.20the.20compiler.20statically.html#187614086">(Feb 07 2020 at 05:07)</a>:</h4>
<p>I think we could still make plugins works by creating proxy methods that take TLS state from the main executable and set it on the plugin.</p>



<a name="187615176"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/compiling%20the%20compiler%20statically/near/187615176" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/compiling.20the.20compiler.20statically.html#187615176">(Feb 07 2020 at 05:38)</a>:</h4>
<p>cc <span class="user-mention" data-user-id="133247">@bjorn3</span> since you might care about this =P</p>



<a name="187615454"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/compiling%20the%20compiler%20statically/near/187615454" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/compiling.20the.20compiler.20statically.html#187615454">(Feb 07 2020 at 05:46)</a>:</h4>
<p>Thanks for pinging me. What is the benifit of completely statically linking rustc? Removing support for hotplug codegen backends would mean that I would have to either build a rustc with support, or I have to integrate cg_clif into rustc, in which case I would have to somehow hack bootstrap to only build a libstd with the resulting compiler, not a whole rustc. In either case the development cycle for cg_cluf gets increased significantly. Also removing <a href="http://librustc_driver.so" target="_blank" title="http://librustc_driver.so">librustc_driver.so</a> would significantly increase the size of rustdoc, miri, clippy and other projects using rustc as library.</p>



<a name="187615739"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/compiling%20the%20compiler%20statically/near/187615739" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/compiling.20the.20compiler.20statically.html#187615739">(Feb 07 2020 at 05:55)</a>:</h4>
<p>Pretty much just performance. It could use ThinLTO, a executable-only TLS model, it would lower startup costs by not having a dylib filled with symbols.</p>
<p>But I do think we could keep hotplug codegen stuff working by just transferring TLS state when calling into the dylib. I haven't come up with a case that wouldn't work with yet</p>



<a name="187636619"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/compiling%20the%20compiler%20statically/near/187636619" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/compiling.20the.20compiler.20statically.html#187636619">(Feb 07 2020 at 12:58)</a>:</h4>
<p><span class="user-mention" data-user-id="116466">@Zoxc</span> I think it would be worthwhile to resolve how much of a performance benefit it'd be. If it's 1-2% I think the costs are probably not worth it.</p>



<a name="187637248"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/compiling%20the%20compiler%20statically/near/187637248" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/compiling.20the.20compiler.20statically.html#187637248">(Feb 07 2020 at 13:06)</a>:</h4>
<p>ThinLTO alone gives -4% on <code>syntex_syntax</code> (probably more if LLVM used it too). I'm waiting on <a href="https://github.com/rust-lang/rust/pull/67759" target="_blank" title="https://github.com/rust-lang/rust/pull/67759">https://github.com/rust-lang/rust/pull/67759</a> to do a perf run.</p>



<a name="187637626"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/compiling%20the%20compiler%20statically/near/187637626" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/compiling.20the.20compiler.20statically.html#187637626">(Feb 07 2020 at 13:12)</a>:</h4>
<p>I'm confused -- can't we already do ThinLTO across most of the compiler? We only ship 1-2 shared libraries today for the compiler, so those could participate in ThinLTO <em>today</em> -- we don't need static linking for that?</p>



<a name="187639242"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/compiling%20the%20compiler%20statically/near/187639242" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/compiling.20the.20compiler.20statically.html#187639242">(Feb 07 2020 at 13:36)</a>:</h4>
<p>I feel like I was talking about this with somebody recently, probably <span class="user-mention" data-user-id="116015">@Alex Crichton</span>? I think we came to the conclusion that <span class="user-mention" data-user-id="116122">@simulacrum</span> suggests -- that we could apply ThinLTO (and PGO) to rustc without going all the way to static linking by having a large library.</p>



<a name="187639280"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/compiling%20the%20compiler%20statically/near/187639280" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/compiling.20the.20compiler.20statically.html#187639280">(Feb 07 2020 at 13:36)</a>:</h4>
<p>Plugins are used by at least servo for their custom GC lint. I think we should devise an alternative for that at some point, but it's not been high on the priority list.</p>



<a name="187639336"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/compiling%20the%20compiler%20statically/near/187639336" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/compiling.20the.20compiler.20statically.html#187639336">(Feb 07 2020 at 13:37)</a>:</h4>
<p>Yes, and we already have that today, at least as far as I can tell. Presumably enabling thinlto and pgo is a matter of ci time and some legwork in bootstrap</p>



<a name="187639355"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/compiling%20the%20compiler%20statically/near/187639355" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/compiling.20the.20compiler.20statically.html#187639355">(Feb 07 2020 at 13:38)</a>:</h4>
<p>In fact I feel like we already do it in some cases</p>



<a name="187640227"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/compiling%20the%20compiler%20statically/near/187640227" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/compiling.20the.20compiler.20statically.html#187640227">(Feb 07 2020 at 13:49)</a>:</h4>
<p>Yes I agree that we do not need a statically linked compiler it the only purpose is to get PGO/LTO/etc benefits, we should start out by applying those techniques to <a href="http://librustc_driver.so" target="_blank" title="http://librustc_driver.so">librustc_driver.so</a> and go from there</p>



<a name="187850876"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/compiling%20the%20compiler%20statically/near/187850876" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/compiling.20the.20compiler.20statically.html#187850876">(Feb 10 2020 at 19:41)</a>:</h4>
<p>so maybe we can just open an issue to track the work that needs to be done here? I did a brief search and it doesn't look like any existing issue covers the desiderata here?</p>



<a name="188826537"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/compiling%20the%20compiler%20statically/near/188826537" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/compiling.20the.20compiler.20statically.html#188826537">(Feb 22 2020 at 17:48)</a>:</h4>
<p>We don't support ThinLTO for dylibs. Not sure if there's any technical limitations or if it's just not implemented. cc <span class="user-mention" data-user-id="124287">@mw</span></p>



<a name="188915185"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/compiling%20the%20compiler%20statically/near/188915185" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/compiling.20the.20compiler.20statically.html#188915185">(Feb 24 2020 at 10:40)</a>:</h4>
<p>I don't know either. The check to not support ThinLTO for Rust dylibs was in there from before I got to work with the code. IIRC, compiling the LLVM backend dylib with xLTO worked fine though.</p>



<a name="188978331"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/compiling%20the%20compiler%20statically/near/188978331" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/compiling.20the.20compiler.20statically.html#188978331">(Feb 24 2020 at 23:42)</a>:</h4>
<p>I removed the check for dylibs + LTO and added <code>Dylib</code> to <code>each_linked_rlib</code> and it seems to just work. I'll do a perf run when LLVM 10 lands.</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>